Microscope system and microscopy method using digital markers

ABSTRACT

A microscope system ( 1 ) includes a microscope and a camera ( 3 ) for generating and recording image-based information on an area of observation. A storage and evaluation unit ( 4 ) is connected to the microscope ( 2 ) and the camera ( 3 ) for detecting parameter settings of the microscope associated with the image-based information. A display ( 5 ) renders visible the digital image of the area of observation, and a control unit ( 6 ) is connected to the storage and evaluation unit ( 4 ) and to the display ( 5 ). The control unit ( 6 ) places digital markers for marking observed objects in a digital image of the area of observation and displays the digital markers on the display ( 5 ). The storage and evaluation unit ( 4 ) selects and stores information on each digital marker so that the marker can be associated at a later point in time with the position of the observed object.

BACKGROUND 1. Field of the Invention

The present invention relates to a microscope system, for example to anoperating microscope system, particularly for neurosurgicalapplications. The invention further relates to a microscopy method, forexample for an operating microscope, particularly for neurosurgicalapplications.

2. Description of the Related Art

Document DE 10 2010 039 289 A1 discloses a microscopy system withoperating microscope for generating a microscopic image, a laserscanning endoscope for generating an endoscopic image, a reflectiondevice for reflecting representations into the microscopic image, andwith a detector that establishes a position of the endoscope andgenerates corresponding positional data. In the framework of thissolution, the laser scanning endoscope must be equipped with anadditional sensor, so that the position of the endoscopy tip relative toa defined reference point in space can be determined, stored and alsocalled up again later. Tissue deformations occurring during theintervention, for example as a result of breathing, removal of tissue,or the like, are not taken into account.

In the document US 2011/0280810 A1, a microscopy system consisting of asurgical microscope and an endomicroscope. In one embodiment, thetemporal monitoring of the endomicroscope is provided by mounting anadditional sensor on the tool tip and/or using commercial navigationsolutions. Here as well, tissue deformations are not taken into account.

Navigation device manufacturers, such as Xion Medical, Brain Lab,Medtronic and others, for example, use the “classic” approach based onexternal navigation devices, for example optical or electromagneticnavigation devices in order to enable digital markers in the operatingfield. For this purpose, however, the surgical instruments requirespecial markers, such as infrared balls, for example.

Previously, surgeons relied on physical markers or on their own memoryin order to mark important points within an operating field or within anarea of observation to be studied microscopically. These points areneeded, for example, in order to mark functional brain regions in aneurosurgical operation, document biopsy removal sites, for example inthe case of a tumor resection, or in order to document positions ofendoscopic recordings. Physical markers have the drawback, however, thatthey cover parts of the tissue for one and, for another, caninadvertently change the position as a result of tissue deformationsand/or surgical activities. Mental markers have the drawback that theyare only “visible” for the surgeon and cannot be documented.

Previously known solutions for digital markers require externalnavigation solutions or the application of special markers. However,tissue deformations cannot be compensated, since changes in the currentarea of observation, for example in the current operating field, are nottaken into account.

It is therefore the object of the present invention to make available anadvantageous microscope system and a corresponding microscopy methodwhich enables the use of digital markers and simultaneously takeschanges in the area of observation to be studied, for example as aresult of movements of the microscope or tissue deformations, intoaccount.

SUMMARY OF THE INVENTION

The microscope system according to the invention comprises a microscopefor generating a microscopic image of an area of observation to bestudied and a camera for recording image-based information of the areaof observation to be studied and for generating a digital image of thearea of observation to be studied. The microscope system furthercomprises a storage and evaluation unit for detecting parameter settingsof the microscope associated with the image-based information, a displayfor displaying the digital image of the area of observation to bestudied and a control unit connected to the storage and evaluation unitand to the display. The control unit is designed to place a number ofdigital markers for marking observed objects in a digital image, forexample in the digital image generated with the aid of the camera, ofthe area of observation to be studied and for displaying the placeddigital markers on the display. Furthermore, the storage and evaluationunit is designed to select or extract and to store information on eachdigital marker so that the marker can be associated at a later point intime with the position of the observed object in the area of observationand displayed in a digital image of the area of observation to bestudied generated at the later point in time in order to mark theposition of the observed object.

In the context of the present invention, i.e., also in relation to themethod according to the invention, the term “information” can, as amatter of principle, also include the positional information to theextent that positional information is present.

The microscope system according to the invention has the advantage thatdigital markers are made available that are robust with respect tochanges in the area of observation, for example in the operating field,particularly as a result of tissue deformations. Moreover, the digitalmarkers that can be generated with the aid of the microscope system arerobust with respect to movements of the microscope, for example of asurgical microscope. A point that is marked in an area of observation tobe studied can thus be found again with the aid of the microscope systemaccording to the invention, even in the event of changes in the area ofobservation, for example as a result of deformations or shifts or due tomovements of the microscope, and reliably associated with the originallymarked point or the originally marked position.

The area of observation to be studied can be an operating area, forexample, particularly a surgical operating area such as a neurosurgicaloperating area, for example, or, more generally, an area ofinvestigation or work or a field being observed.

In principle, the camera can be connected to the microscope. Forexample, the camera can be integrated into the microscope. It cantherefore be ab internal camera. The microscope used is preferable anoperating microscope, for example a surgical, particularly aneurosurgical operating microscope or an endomicroscope or a microscopefor the removal of biopsies.

Due to the fact that, with the aid of the microscope system according tothe invention, image-based information on the area of observation to bestudied, for example the operating field, and, in addition, parametersettings of the microscope can be used, no physical markers, for examplecolor markings, need be applied to the deformable tissue. Moreover, noadditional external sensors need to be mounted or external navigationsolutions used or existing navigation solutions adapted. For instance,the image-based information on an operating field can be recorded by aninternal camera of a surgical microscope and parameter settings of themicroscope simultaneously stored. The storage and evaluation unit ispreferably designed to detect the parameter settings zoom and/or focusof the microscope.

The inventive image-based recognition of markers in the deformabletissue does make it possible in principle not to mount any additionalsensors or markers. However, if navigation data, particularly therelative or absolute position of the surgical microscope in space to thepatient, are present—whether through an external solution or an internalsolution in which the surgical microscope itself recognizes ordetermines how it has moved over time in space (by means of integratedsensors on the microscope axes, for example), so that it thereforedetermines and optionally stores the relative change between two imagerecordings—this information can be used to support the image-basedrecognition. For example, if the respective viewing angle to the fieldis known, the searching space can be better delimited.

The camera used can be designed, for example, to record image-basedinformation as a video image of the area of observation being observedthrough the microscope. The digital image of the area of observation cantherefore be a video image of the area of observation being observedthrough the microscope taken by the camera, for example.

Moreover, the control unit for setting and displaying a number ofdigital markers can comprise a touchscreen and/or a device for gesturecontrol and/or a device for speech control. The device for gesturecontrol can be designed such that it enables control by finger, hand orinstruments, such as surgical instruments. With the aid of the controlunit for setting and displaying a number of digital markers, digitalmarkers can be placed, for example, in the digital image of an operatingfield, for example in the video image of a surgical microscope.

The control unit for setting and displaying the digital markers ispreferably designed so as to place and display digital markers in theform of points and/or lines and/or circles and/or polygons and/orfreeform contours. This enables versatile and individualized structuringof the markers used appropriate to the respective situation.

Moreover, the control unit for setting and displaying the digitalmarkers is preferably designed to show and hide the digital markers inthe display. This makes it possible to make the area covered by themarker visible as desired. In this way, tissue lying under the marker isnot covered.

What is more, the display for displaying the digital markers in thedigital image of the area of observation can be embodied by an ocular ofthe microscope. This makes it possible for the placed markers to bevisible directly through the ocular of the microscope, i.e., without theneed for an additional display outside of the microscope. In addition oralternatively, the display for displaying the digital markers in thedigital image of the area of observation can be embodied as a displayunit arranged externally from the microscope. This makes it possible,for example, for a plurality of people to observe the placed markers andmonitor them as desired.

The storage and evaluation unit is embodied, for example, such that theposition of the respective marker in the area of observation can beassociated again at a later point in time with the position of theoriginally marked observed object in the area of observation and/or theposition of the marker and hence of the marked observed object in thearea of observation can be followed over time. This offers the advantagethat changes in the position of the observed object in the area ofobservation, for example as a result of a deformation of observed tissueor due to tissue removal, can be taken into account with respect to thecurrent position of the marked observed object.

The microscope system also has a navigation unit for detecting kinematicpositional information of the microscope. The navigation unit can beintegrated into the microscope or arranged externally. The navigationunit for detecting kinematic positional information of the microscopecan particularly be designed to determine the viewing angle to the areaof observation, for example an operating area, and/or to determine anabsolute position of the microscope, for example the initial position.

What is more, the microscope system can comprise a unit for detectingtopographic information over the area of observation. This can beachieved, for example, by means of a 3D sensor or a stereoscopicrecording unit. In this way, the height profile in the vicinity of theobserved object of the markers is made more readily identifiable and canbe better followed.

The storage and evaluation unit is preferably designed to analyze imageinformation in the vicinity of the respective digital marker with theaid of feature descriptors and describe it on the basis ofcharacteristic features. This can be achieved, for example, through theuse of the feature descriptors commonplace in the computer visioncommunity, such as SIFT, SURF, BRIEF, ORB, or other feature descriptors.In this case, the storage and evaluation unit is designed for the use ofthe cited feature descriptors. In addition, the storage and evaluationunit is preferably designed to save parameter settings of themicroscope, for example of the surgical microscope, together with therespective marker. For example, the focus and/or zoom of the microscopecan be saved with the respective marker in order to calculate thecurrent image enlargement, so that the zoomed area can be adjusted atanother setting accordingly. If present, the navigation data can also bestored in the storage and evaluation unit; for example, they can bestored together with the marker. As a result, it is possible, forexample, to move again to the spatial position when the correspondingmarker is selected. Moreover, if the microscope has moved, the change inviewing angle can be calculated and this information can be taken intoaccount in the image-based recognition of the markers.

In the context of the microscopy method according to the invention, amicroscopic image of an area of observation to be studied is generatedwith a microscope. Using a camera, for example using a camera connectedto the microscope, image-based information on the area of observation tobe studied is recorded, and a digital image is generated of the area ofobservation to be studied. Furthermore, parameter settings of themicroscope associated with the image-based information are detected, forexample with the aid of a storage and evaluation unit connected to themicroscope and the camera. The digital image of the area of observationto be studied is rendered visible, for example with the aid of adisplay. A number of digital markers are placed in order to markobserved objects in a digital image of the area of observation to bestudied and displayed. The markers can be set with the aid of a controlunit, for example.

The digital image in which the markers are placed and displayed can be adigital image generated with the aid of a camera, for example. Thecontrol unit can be connected to a storage and evaluation unit and to adisplay.

Moreover, in the context of the microscopy method according to theinvention, information is selected and extracted and stored for eachdigital marker so that the marker can be associated at a later point intime to the position of the observed object in the area of observation.This can be done with the aid of a storage and evaluation unit, forexample. Finally, in the context of the method according to theinvention, the digital marker is displayed in a digital image of thearea of observation to be studied generated at a later point in time inorder to mark the position of the observed object.

In principle, the microscopy method according to the invention has thesame advantages as the microscope system described previously. Themethod according to the invention can particularly be carried out withthe aid of a microscope system described previously. A previouslydescribed microscope system can therefore be used in the context of themicroscopy method according to the invention.

In the method, for example, the zoom and/or focus of the microscope canbe detected by a storage and evaluation unit, for example. Preferably,zoom and/or focus of the microscope are detected continuously. The citedparameter settings of the microscope can be associated with set digitalmarkers. If present, navigation in the storage and evaluation unit canalso be stored in this context; for example, they can be stored togetherwith the marker. As a result, it is possible to move again to thespatial position when the corresponding marker is selected. Furthermore,insofar as the surgical microscope has moved, the change in viewingangle can be calculated and this information can be taken into accountduring the image-based recognition of the markers.

Preferably, the area of observation is evaluated continuously by meansof image processing algorithms, and the position of the digital markersis adapted to the position of the observed object in the event ofchanges such as deformations, particularly tissue deformations, of theobserved object in the area of observation and/or in the event that themicroscope is moved. For example, the position of the markers isadjusted, preferably continuously, to the position of the observedobject using tracking algorithms.

Moreover, image processing algorithms can be used to continuouslyevaluate an operating area or an operating field, particularly andspecifically the recorded video stream of a surgical microscope, so thatthe positions of a set marker can be adapted to a change in the positionof the marked observed object. Possible changes in the position of themarked observed object can be caused by tissue deformations and/or bymovement of the surgical microscope.

Tracking algorithms from the field of computer vision can be used forthe adjustment. For example, methods can be used here that are based onthe optical flow, such as a Kanade-Lucas-Tomasi feature tracker, andoptionally hybrid trackers that combine the optical flow with featuredescriptors in order to increase the robustness of the tracker (forexample, see Lee, Taehee and Höllerer, Tobias, Multithreaded HybridFeature Tracking for Markerless Augmented Reality, IEEE Transactions onVisualization and Computer Graphics, Vol. 15, No. 3, May/June 2009 andhttp:/dx.doi.org/10.1109/TVCG.2008.190).

Since changes in the zooming and focusing of the surgical microscope canchange the image enlargement and thus the search area, the parametersare preferably read out continuously and used by the tracking algorithm.Optionally, the tracking algorithm can be supported by the additionalinformation sources already mentioned previously; particularly,additional parameter settings of the surgical microscope, positionalinformation, topographic information, etc., in order to enable betterdelineation of search areas, for example, particularly place and scale,or to make more information available for the feature descriptors,particularly topographic features.

Moreover, through the use of tracking algorithms for each marker, it ispossible to determine a probability of its positional accuracy, throughcalculation, for example. This probability of the positional accuracy ofthe respective marker can be stored together with the marker as well ason an output unit, for example a display.

Furthermore, information for the determination of an expiration time ofthe respective marker can be established for a marker. The marker canthus be associated with a kind of “expiry time” or “expiration date” or“use-by period.” What is more, information for determining thepositional accuracy of a marker can be established for it. This can bestored together with the marker. For example, a marker can be declaredor characterized as invalid or be deleted if an established accuracythreshold for its positioning has been exceeded or if its positioning isno longer possible within an established accuracy threshold. This can bethe case with heavy tissue resections, for example.

In the context of the microscopy method according to the invention, thedigital markers can be rendered visible or displayed by an ocular of themicroscope. Alternatively or in addition, the digital markers can berepresented or displayed by at least one display unit or output unitarranged externally to the microscope. Moreover, the digital markers canbe shown and hidden as needed. This makes it possible to examine areasof observation that might be covered by the marker.

In addition, a probability for the positional accuracy of the respectivemarker can be determined for markers. The marker can be associated withthe determined probability. If a probability for the positional accuracyof the respective marker has been determined for markers, a filter canalso be used in order to show only markers with a minimum probability.

Optionally, a marker can be selected and the position of the observedobject sighted through the microscope. This makes is possible to selectindividual markers and to focus through the microscope, for example thesurgical microscope, in a targeted manner. The kinematic characteristicsof the microscope can be exploited in order to first move to the desiredlocation, i.e., the spatial position, whose navigation data wereoriginally detected with the marker before it is sighted and optionallyfocused on. This makes it possible to locate the originally markedpositions quickly and precisely.

Moreover, the placed markers can optionally be permanently storedtogether with the image information for documentation purposes andtransferred to a data management system such as PACS, for example.

The inventive microscope system and the inventive microscopy methodenable the use of image-supported information for the creation of robustdigital markers—e.g., which are insensitive in relation to tissueinformation—for a microscope, preferably a surgical microscope, withoutthe use of additional navigation solutions or external markers.Consequently, no physical markers, such as color markings in thedeformable tissue, need to be applied. Moreover, no additional externalsensors need to be put in place or external navigation solutions used orexisting navigation solutions adapted.

The inventive image-based image recognition of markers in deformabletissue does make it possible in principle not to mount any additionalsensors or markers. However, insofar as navigation data, particularlythe relative or absolute position of the surgical microscope in thespace to the patient, are present—whether through an external solutionor an internal solution in which the surgical microscope itselfrecognizes or determined how it has moved over time in space, so that itthus determines and optionally stores the relative change between tworecorded images—this information can be used to support the image-basedrecognition. For example, if the respective viewing angle to the area isknown, the search space can be better delimited.

Moreover, the use of specific characteristics of the microscope,particularly of the surgical microscope, is enabled for improvedtracking of the markers. In addition, image-supported digital markerscan be integrated into the microscopic or surgical procedure with theaid of the present invention.

Additional features, characteristics and advantages of the presentinvention are described in further detail below on the basis ofexemplary embodiments with reference to the enclosed figures. All of thefeatures described above and in the following are advantageous bothalone and in any combination with each other. The exemplary embodimentsdescribed below merely constitute examples that do not limit the subjectmatter of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic view of a microscope system according to theinvention.

FIG. 2 shows a schematic view of an example of a method according to theinvention in the form of a flowchart.

DETAILED DESCRIPTION

One example of a microscope system according to the invention isexplained in further detail below with reference to FIG. 1. FIG. 1 showsa schematic view of a microscope system 1 according to the invention. Itcomprises a microscope 2, a camera 3, a storage and evaluation unit 4, adisplay 5 and a control unit 6. The microscope 2 can be a surgicalmicroscope or an operating microscope or an endomicroscope or amicroscope for use in the context of biopsy removal.

The microscope 2 is connected to the camera 3. For this purpose, thecamera 3 can preferably be integrated into the microscope 2.Alternatively, the camera 3 can be arranged externally from themicroscope 2. The display 5 is connected to the microscope 2 and thecamera 3. The display 5 can be embodied as a display unit arrangedexternally from the microscope 2 and/or camera 3. Alternatively, thedisplay 5 can be integrated into the microscope 2, for example in orderto enable a display through the ocular of the microscope.

Moreover, the microscope 2, the display 5 and the camera 3 are connectedto the storage and evaluation unit 4. The storage and evaluation unit 4can be embodied, for example, as a CPU or GPU or comprise a CPU or GPU.The storage and evaluation unit 4 is particularly designed to receiveinformation or data from the microscope 2 and the camera 3, evaluate oranalyze it and output it to the display 5.

The control unit 6 is connected to the display 5 and the storage andevaluation unit 4. With the aid of the control unit 6, signals can betransferred to the display 5 and to the storage and evaluation unit 4.For example, digital markers can be placed in digital images recorded bythe camera and rendered visible via the display 5. At the same time, theset digital markers can be stored an analyzed together with additionalinformation by the storage and evaluation unit 4.

Alternatively to the arrangement shown in FIG. 1, the microscope 2and/or the camera 3 can be connected indirectly to the display 5, beingconnected to the display 5, exclusively via the storage and evaluationunit 4, for example.

The inventive method will now be explained in further detail withreference to FIG. 2. Schematic FIG. 2 shows an example of a methodaccording to the invention in the form of a flowchart. In a first step11, a microscopic image of an area of observation to be studied isgenerated with a microscope 2 and image-based information on the area ofobservation to be studied is recorded with a camera 3. In this context,a digital image is generated of the area of observation to be studied.

In a second step 12, which can be performed simultaneously with thefirst step 11, parameter settings of the microscope 2 associated withthe image-based information are detected. This can be achieved with theaid of a storage and evaluation unit 4 connected to the microscope 2 andthe camera 3.

In another step 13, which can be performed simultaneously with one orboth of the abovementioned steps 11 and/or 12, the digital image of thearea of observation to be studied is rendered visible, preferably withthe aid of a display 5.

Then, in a next step 14, a number of digital markers, i.e., at least onemarker, for marking observed objects, i.e., for marking at least oneobserved object, is placed in the digital image of the area ofobservation to be studied and displayed. The placed markers can berendered visible with the aid of the previously mentioned display 5, forexample.

In another step 15, information on each digital marker is selected orextracted and stored so that the marker can be associated at a laterpoint in time with the position of the observed object in the area ofobservation. In a next step 16, the digital marker is displayed in adigital image of the area of observation to be studied generated at alater point in time in order to mark the position of the observedobject. A digital image of the area of observation to be studied is thusgenerated at a later point in time and the position of the observedobject in the area of observation to be studied is displayed at a laterpoint in time. This can preferably be achieved with the aid of a storageand evaluation unit 4.

In principle, the method according to the invention can compriseadditional method steps. For example, the digital markers can be shownand hidden so that the underlying area of observation—tissue forexample—is not covered. The digital markers can be embodied in the formof points, lines, polygons or freeform contours, for example. They canbe placed in the digital image of the area of observation, for examplean operating field, specifically in the video image of the surgicalmicroscope, by means of a touchscreen, through gesture control, forexample with the aid of fingers, the hand or surgical instruments, or bymeans of speech control.

A series of information is extracted and stored for each marker, so thatthe marker can later be associated again with a position of an observedobject in the area of observation, for example in an operating areaand/or followed over time. For this purpose, image information in theclose proximity of the markers is analyzed (so-called featuredescriptors) and described using characteristic features. This can bedone using features descriptors that are commonplace in the computervision community, such as SIFT, SURF, BRIEF, ORB or others.

The parameter settings of the microscope, for example of the surgicalmicroscope, are stored together with the marker. For example, these canbe the focus and/or zoom of the microscope in order to calculate thecurrent image enlargement, for example, thus enabling adaptationaccording to the search area at another setting. Optionally, kinematicpositional information of the microscope system can also be storedintrinsically or via external navigation solutions, for example. Thiscan be done, for example, in order to determine the viewing angle to thearea of observation, for example to the operating area, and in order todetermine an initial absolute position.

Moreover, topographic information of the area of observation, forexample of the operating area, can also be detected and stored in orderto enable better recognition and monitoring of the marker through theheight profile in the vicinity of the marker. This can be achieved, forexample, by a 3D sensor or stereoscopic recordings.

The area of observation, for example the operating area, specificallythe video stream of the surgical microscope, is continuously evaluatedby means of image processing algorithms so that the positions of themarkers can be adapted to changes in the vicinity of the marker. Forexample, the position of the markers can be adapted in relation tochanges caused by tissue deformations and/or move of the (for examplesurgical) microscope.

Tracking algorithms from the area of computer vision are used duringevaluation and adjustment. For example, they can involve methods thatare based on the optical flow, such as a Kanade-Lucas-Tomasi featuretracker, and optionally hybrid trackers that combine the optical flowwith feature descriptors in order to increase the robustness of thetracker. One example of this can be found in Lee, Taehee and Höllerer,Tobias, Multithreaded Hybrid Feature Tracking for Markerless AugmentedReality, IEEE Transactions on Visualization and Computer Graphics, Vol.15, No. 3, May/June 2009.

Since changes in the zooming and focusing of the surgical microscope canchange the image enlargement and thus the search area, the parametersare preferably read out continuously and used by the tracking algorithm.Optionally, the tracking algorithm can be supported by the additionalinformation sources already mentioned previously, such as additionalparameter settings of the surgical microscope, positional information,topographic information, etc., in order to enable better delineation ofsearch areas, for example place and scale, or to make more informationavailable for the feature descriptors, for example topographic features.

Optionally, the tracking algorithm calculates a probability ofpositional accuracy for each marker which is stored together with themarker and can also be displayed on an output unit. Also optionally,markers can be provided with a “use-by date” or expiration time orperiod of validity. The markers can be declared invalid if positioningis no longer possible or lies beyond an accuracy threshold. For example,this can be caused by heavy tissue resections in relation to a surgical,for example a neurosurgical, operation.

The digital markers can be represented via the ocular and/or an externaldisplay unit 5 or several external display units and can be shown andhidden as needed. If the markers are provided with probabilities, afilter can optionally be used and activated in order to show onlymarkers with a minimum probability.

Optionally, individual markers can be selected and sighted through themicroscope. The individual markers can particularly be focused onthrough the surgical microscope 2 in a targeted manner. If available,the microscope, preferably the surgical microscope, can exploit itskinematic characteristics in order to first move to the desired locationbefore focusing. Optionally, the placed markers can be permanentlystored together with the image information and transferred to a datamanagement system, for example PACS, for documentation purposes.

Various areas of application, namely an application in relation to anendomicroscope, a biopsy and the marking of functional areas of thebrain during tumor resection are named and briefly explained below.

With regard to endomicroscopy, a surgeon can additionally use anendomicroscope in the visual field of the surgical microscope forcellular diagnosis, when an endoscopic image is taken, theendomicroscope can be automatically detected in the image field of thesurgical microscope, a marker placed on the tip of the endomicroscope,and the recorded endoscopic image linked to the marker. The marker canbe monitored over the further course of the operation. By clicking onthe marker on a touchscreen of the surgical microscope at its currentposition, the associated recorded endoscopic image can be called up atany time.

With regard to biopsies, several minutes typically pass between theremoval of the biopsy and the pathologist's report. However, since thesurgeon may be continuing with the operation, they will want to knowretrospectively when the pathology report comes in where the specimenswere taken. For this reason, the place in the image at which the biopsyis taken is marked during removal of the biopsy on the touchscreen ofthe surgical microscope. Optionally, a textual label and/or ID can alsobe added to the marker. The placed markers are monitored over thefurther course of the operation, so that, when the pathology reportarrives, the surgeon can select the stored markers via a menu system inthe touchscreen and have the surgical microscope focus on themautomatically.

With regard to tumor resections in the brain, functional areas orfunctional areas of the brain can be determined through stimulationafter the craniotomy and digital markers can be placed for the variousfunctional regions, for example through marking on site with theautomatically detected tool tip and placing the marker using a footswitch. These markers can be monitored over the further course of thetumor resection, so that the functional areas can continue to besuperimposed over the operating area as markers despite possible tissuedeformations and removal of tissue.

LIST OF REFERENCE SYMBOLS

1 microscope system

2 microscope

3 camera

4 storage and evaluation unit

5 display

6 control unit

11 Generation of a microscopic image and recording of image-basedinformation

12 Detection of parameter settings of the microscope and associationwith the image-based information

13 Displaying of the digital image

14 Placement and displaying of digital markers

15 Selection and storage of information associated with the marker

16 Displaying of the marker at a later point in time to mark theposition of the observed object

What is claimed is:
 1. A microscope system comprising: a microscope forgenerating a microscopic image of an area of observation to be studied;a camera for recording image-based information on the area ofobservation to be studied and for generating a digital image of the areaof observation to be studied; a storage and evaluation unit connected tothe microscope and the camera for detecting parameter settings of themicroscope associated with image information, a display for renderingvisible the digital image of the area of observation to be studied, andan input device connected to the storage and evaluation unit and to thedisplay, the input device configured to embed digital markers formarking observed objects in the digital image when the digital image isdisplayed on the display, wherein the storage and evaluation unit isdesigned to select and store information on each digital marker toassociate the digital marker with a position of the observed object inthe area of observation and in the digital image of the area ofobservation at a later point in time, the digital marker is embedded ata location in the digital image characterized by the image information,and wherein the storage and evaluation unit is configured to evaluatethe area of observation periodically by image processing algorithms andsynchronize the position of the digital markers to the position of theobserved object in an event of changes in the observed object in thearea of observation or in an event of movement of the microscope,wherein the storage and evaluation unit is configured to analyze theimage information contained in the digital image in the vicinity of arespective marker describe the image information with respect tocharacteristic features, associate the characteristic features with therespective marker and store the association such that it is retrievable;wherein the storage and evaluation unit is configured to perform atleast one of the following: detect at least one of: parameter settingsof zoom and a focus of the microscope; associate, again at a later pointin time, the position of the digital marker in the area of observationwith at least one of the position of the originally marked observedobject in the area of observation and the position of the marker andhence of the marked observed object in the area of observation can befollowed over time; and analyze image information in the vicinity of therespective digital marker with the aid of feature descriptors anddescribe it on the basis of characteristic features.
 2. The microscopesystem of claim 1, wherein the microscope is an operating microscope oran endomicroscope or a microscope for biopsy removal.
 3. The microscopesystem of claim 1, wherein the input device for placing and displaying anumber of digital markers comprises a touchscreen and/or a device forgesture control and/or a device for speech control, and/or the inputdevice for placing and displaying a number of digital markers isdesigned so as to embed and display digital markers in the form ofpoints and/or lines and/or circles and/or polygons and/or freeformcontours, and/or the input device for setting and displaying the digitalmarkers is designed to show and hide the digital markers in the display.4. The microscope system of claim 1, wherein the display for renderingvisible the digital markers in the digital image of the area ofobservation display is embodied by an ocular of the microscope and/orthe display is embodied as a display unit arranged externally from themicroscope.
 5. The microscope system of claim 1, further comprising anavigation unit for detecting kinematic positional information of themicroscope.
 6. The microscope system of claim 1, further comprising aunit for detecting topographic information on the area of observation.7. A microscopy method, wherein a microscopic image of an area ofobservation to be studied microscopically is generated with amicroscope, and image-based information on the area of observation to bestudied microscopically is recorded with a camera and a digital image ofthe area of observation to be studied is generated, the methodcomprising: parameter settings of the microscope associated with theimage-based information; rendering visible the digital image of the areaof observation to be studied is rendered visible, embedding a number ofdigital markers in the digital image of the area of observation to bestudied and displayed, selecting and storing information on each digitalmarker so that the marker can be associated at a later point in timewith the position of the observed object in the area of observation,displaying the digital marker in a digital image of the area ofobservation to be studied generated at a later point in time in order tomark the position of the observed object, and evaluating the area ofobservation periodically by image processing algorithms andsynchronizing the position of the digital markers to the position of theobserved object in an event of changes in the observed object in thearea of observation or in an event of movement of the microscope, andanalyzing the image information contained in the digital image in thevicinity of a respective marker describe the image information withrespect to characteristic features, associating the characteristicfeatures with the respective marker and storing the association suchthat it is retrievable; wherein the storage and evaluation unit isconfigured to perform at least one of the following: detecting at leastone of: parameter settings of zoom and a focus of the microscope;associating, again at a later point in time, the position of the digitalmarker in the area of observation with at least one of the position ofthe originally marked observed object in the area of observation and theposition of the marker and hence of the marked observed object in thearea of observation can be followed over time; and analyzing imageinformation in the vicinity of the respective digital marker with theaid of feature descriptors and describe it on the basis ofcharacteristic features.
 8. The microscopy method of claim 7, whereinthe position of the markers is adapted to the position of the observedobject using tracking algorithms.
 9. The microscopy method of claim 7,wherein a marker is selected and the position of the observed object issighted through the microscope.